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OVERVIEW 


How  critical  is  the  role  of  Federal  First-line  supervisors?  They  are  responsible  for  providing  the  front-line  management 
of  most  Government  functions,  and  are  key  to  ensuring  that  efficient  and  effective  use  is  made  of  the  Federal  work 
force  (one  of  the  merit  principles  governing  the  civil  service  system).  The  U.S.  Merit  Systems  Protection  Board 
(MSPB)  is  very  interested  in  determining  how  effectively  Federal  agencies  fill  these  positions.  Recently,  the  Board 
studied  first-line  supervisory  selection  systems  in  the  Federal  Government  to  determine  whether  agency  needs  are 
being  met.  The  study  focused  on  identifying  typical  Federal  supervisory  selection  practices  as  well  as  alternative 
approaches  to  supervisory  selection. 

The  study  revealed  that: 

■  Because  of  the  diversity  throughout  the  Federal  sector  in  the  work  performed,  organizational  environments,  and 
agency  missions,  it  is  highly  unlikely  that  any  one  strategy  for  selecting  first-line  supervisors  will  be  totally  effec¬ 
tive  in  meeting  all  needs. 

■  Currently,  Federal  agencies  use  few  different  or  innovative  systems  tailored  specifically  for  selecting  first-line 
supervisors.  For  the  most  part,  agencies  select  individuals  for  first-line  supervisory  positions  using  a  process 
identical  to  the  one  they  use  for  all  other  types  of  jobs. 

■  At  two  representative  sites  MSPB  visited,  where  the  typically  used  system  is  in  operation,  managers  are  generally 
satisfied  that  it  meets  their  needs.  However,  data  from  an  MSPB  questionnaire  survey  and  other  sources  caution 
against  concluding  that  the  typical  strategy  is  always  effective. 

■  Of  the  agencies  which  said  they  use  the  typical  system,  one  out  of  four  also  indicated  that  their  organizations  need 
better  methods  for  identifying  candidates  for  first-line  supervisory  positions. 

■  The  selection  strategy  typically  used  by  most  agencies  may  not  be  adequate  for  meeting  selection  needs  in  all 
situations.  A  few  agencies  have  in  fact  developed  alternative  selection  strategies  which  can  deal  effectively  with 
the  needs  arising  from  these  situations. 

■  OPM  has  not  evaluated  Govcmmentwide  experience  with  the  CSRA  requirement  that  all  new  first-line  supervisors 
serve  a  probation  period  as  the  final  step  in  the  total  selection  process. 

This  report  recommends  that: 

■  Agencies  lake  a  much  closer  look  at  their  own  supervisory  selection  strategics  and  determine  if  they  arc  adequate 
for  meeting  their  individual  selection  needs.  Those  agencies  that  are  experiencing  organizational  problems  which 
may  be  related  to  the  quality  of  their  supervisors  are  especially  encouraged  to  consider  alternative  selection 
methods. 

■  OPM  undertake  a  thorough  evaluation  of  the  use  of  supervisory  probationary  programs  by  Federal  agencies. 

■  OPM  exercise  leadership  in  identifying  and  disseminating  information  about  effective  alternatives  in  the  critical 
area  of  supervisory  selection. 
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INTRODUCTION 


WHY  MSPB  STUDIED  SUPERVISORY 
SELECTION 


Imagine  a  job  in  which  you  arc  expected  to  motivate 
others  to  the  highest  levels  of  productive  and 
efficient  performance  in  pursuit  of  the  organiza¬ 
tion’s  goals,  yet,  at  the  same  time,  be  concerned 
with  the  unique  needs  of  those  whose  performance 
you  arc  supposed  to  motivate,  even  when  those 
needs  are  sometimes  in  conflict  with  the  overall 
goals  of  the  organization.  Impossible  task?  For 
over  160,000'  Federal  employees  who  occupy  the 
position  of  first-line  supervisor,  this  scenario  is 
played  out  every  day  on  the  job. 

In  every  agency,  the  role  of  first-line  supervisor  is 
critical.  Because  first-line  supervisors  arc  respon¬ 
sible  for  providing  the  front-line  management  of 
most  Government  functions,  these  individuals  arc 
key  to  ensuring  that  efficient  and  effective  use  is 
made  of  the  Federal  work  force  (one  of  the  merit 
principles  governing  the  civil  service  system).  Not 
only  arc  they  responsible  for  assigning,  directing, 
and  evaluating  the  work  of  subordinates,  but,  as  the 
link  between  employees  and  upper  management, 
they  must  communicate  the  organization’s  mission 
and  goals  to  employees  in  a  meaningful  way  that 
will  motivate  them  toward  effective  performance. 
The  most  challenging  aspect  of  this  position  may  be 
that  even  though  the  supervisor  is  familiar  with  the 
technical  work  of  the  unit,  the  supervisory  duties 
which  are  so  critical  to  the  organization’s  perform¬ 
ance  arc  often  completely  foreign  to  new  supervi¬ 
sors.  Thus,  the  part  of  the  job  which  is  most  critical 
from  an  organizational  standpoint  is  also  the  most 
difficult  for  a  new  supervisor  to  perform. 


1  Source:  U.S.  Office  of  Personnel  Management,  extracted 
from  the  September  1987  "Current  Status  Report”  for  all  agencies 
except  the  U.S.  Postal  Service 


Unfortunately,  such  a  scenario  is  rife  with  potential 
for  error.  Mistakes  made  by  supervisors  arc  poten¬ 
tially  much  worse  and  more  costly  than  those  made 
by  nonsupervisory  employees.  Undeniably,  first- 
line  supervisors  are  central  to  gelling  the  maximum 
effort  from  their  work  groups.  The  actions  they 
take,  whether  positive  or  negative,  arc  amplified 
through  their  subordinate  staffs.  Consider  the 
effects  that  a  supervisor  who  performs  poorly  can 
have  on  the  work  group  as  a  whole: 

-  Low  morale  among  subordinates 

—  Low  productivity  of  work  group 

-  Low  quality  products  or  services 

—  Frequent  grievances  or  complaints  by 
subordinates  or  clients 

—  High  absenteeism  or  turnover  by 
subordinates 

Even  if  an  organization  tries  to  improve  a  deficient 
supervisor’s  performance  through  remedial  train¬ 
ing,2  the  organization  still  endures  some  short-term, 
if  not  permanent,  negative  effects.  Clearly  then,  an 
organization  must  pay  attention  to  how  well  its 
supervisory  selection  system  is  working;  there  is  too 
much  at  stake  to  tolerate  a  selection  system  which  is 
not  helping  to  identify  high  performers  for  those  key 
jobs. 

Because  of  the  critical  role  first-line  supervisors  play 
in  organizations,  the  U.S.  Merit  Systems  Protection 
Board  (MSPB)  is  very  interested  in  determining  the 
effectiveness  with  which  Federal  agencies  fill  these 
positions.  Recently,  MSPB  studied  white-collar 


2 

In  his  book  "Improving  Supervisors'  Effectiveness"  Jack  J 
Phillips  observes  lhat  "(t)raining  can  compensate  for  some 
deficiencies,  bul  a  few  employees  will  never  be  effective  as 
supervisors,  no  matter  how  much  training  they  rcceivc***at  best  it 
can  bring  slightly  below  average  employees  up  to  satisfactory 
levels."  (Josscy-Bass,  1985,  p,  30.) 
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first-line  supervisory  selection  systems  in  the 
Federal  Government  to  determine  whether  critical 
needs  are  being  met  in  this  area.  To  meet  our  study 
objective  wc  focused  on  identifying  typical  supervi¬ 
sory  selection  practices  as  well  as  innovative 
approaches  to  supervisory  selection,  rather  than  on 
assessing  the  legal  or  regulatory  correctness  of 
individual  selection  actions. 

HOW  SUPERVISORY  SELECTION  DIFFERS 
FROM  NONSUPERVISORY  SELECTION 


Selecting  an  employee  for  his  or  her  first  supervi¬ 
sory  job  can  be  a  formidable  challenge  because  it 
requires  assessing  that  individual’s  potential  for 
performing  tasks  often  unlike  those  the  employee 
previously  performed.  Also,  there  is  no  one  model 
of  supervisory  performance  to  use  as  a  standard  in 
all  situations. 

The  challenge  of  assessing  applicants’  “supervisory 
potential”  is  well  known  to  Federal  personnel 
practitioners.  For  the  most  part,  there  is  little 
operational  guidance  on  Federal  supervisory 
selection.  General  guidance  provided  for  filling  all 
types  of  Federal  jobs  focuses  on  evaluating  appli¬ 
cants’  prior  training  and  experience  to  predict  future 
performance  While  this  approach  may  provide 
accurate  predictions  of  performance  for  many  types 
of  jobs,  using  it  to  evaluate  first-time  applicants  for 
first-line  supervisory  jobs  is  more  problematic.  This 
is  because  much  of  the  work  performed  in  the 
supervisory  job  is  very  different  from  that  done  in 
the  nonsupervisory  job,  yet  the  training  and  experi¬ 
ence  information  for  first-line  supervisory  job 
applicants  is  typically  based  on  their  previous 
nonsupervisory  jobs.  In  short,  the  problem  for  those 
evaluating  applicants  for  first-line  supervisory  jobs 
becomes  how  to  assess  a  person’s  ability  to  perform 
certain  types  of  duties  if  she  or  he  has  never  had  the 
opportunity  to  demonstrate  such  abilities  before. 
Later  in  this  report,  we  discuss  how  a  few  agencies 
have  developed  strategics  for  assessing  the  knowl¬ 


edge,  skills,  and  abilities  of  applicants  for  supervi¬ 
sory  jobs  without  the  reliance  on  information  about 
their  training  and  experience. 

In  addition  to  problems  associated  with  relying  on 
previous  training  and  experience  to  evaluate 
candidates,  there  is  also  the  problem  of  identifying 
which  particular  knowledge,  skill,  or  ability  to 
assess  to  answer  the  question  of  what  constitutes 
“supervisory  potential.”  If  a  single  model  of  an 
ideal  supervisor  existed,  it  would  be  possible  to 
develop  a  standardized  approach  for  assessing  what 
supervisors  need  to  perform  successfully.  However, 
“good”  supervision  is  situational,  inasmuch  as 
characteristics,  expectations,  and  values  of  the 
individual  organization  help  determine  what  is 
effective.  Therefore,  while  each  supervisory  job 
generally  requires  a  combination  of  technical  and 
leadership  proficiency,  specific  skills,  knowledge,  or 
abilities  and  their  relative  importance  can  vary 
widely.  Our  review  of  agencies’  practices  revealed 
that  some  agencies  have  made  a  concerted  effort  to 
systematically  identify  appropriate  skills,  knowl¬ 
edge,  abilities,  or  other  criteria  for  use,  as  well  as 
the  best  mix  of  technical  and  supervisory  competen¬ 
cies. 

HOW  WE  CONDUCTED  THE  STUDY 


To  begin  our  review  of  supervisory  selection  in  the 
Federal  Government,  we  asked  the  22  largest 
Federal  agencies  to  provide  us  information  about 
any  methods,  techniques,  forms,  or  systems  they 
currently  use  in  selecting  white-collar  (GS/GM) 
first-line  supervisors.  Eighteen  agencies  provided 
information  in  response  to  our  request.3  From 


3  We  received  information  from  the  Departments  of  Agricul¬ 
ture,  Airl-orcc,  Army,  Commerce,  Education,  Energy,  Health  and 
Human  Services,  Interior,  Justice,  Labor,  Navy,  State,  Transporta¬ 
tion,  and  Veterans  Affairs;  General  Services  Administration; 
National  Aeronautics  and  Space  Administration;  Small  Business 
Administration;  and  the  Office  of  Personnel  Management. 
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materials  submitted,  it  was  clear  that  most  agencies 
used  the  same  general  approach  to  supervisory 
selection.  We  selected  for  further  study  one 
agency’s  submission  as  representing  this  “typical 
approach”  to  supervisory  selection.  We  also 
selected  the  four  agencies’  submissions  that  re¬ 
flected  innovative  approaches  to  Federal  supervisory 
selection,  also  to  be  examined  more  closely. 

Our  closer  examination  of  the  five  agencies’ 
programs  involved  collection  of  detailed  information 
directly  from  both  agency  employees  and  agency 
records,  and  discussions  with  program  officials. 
Agency  records  provided  summary  data  concerning 
performance  of  supervisors  selected  under  these 
programs.  We  conducted  onsite  structured  inter¬ 
views  with  a  sample  of  first-line  supervisors  who 
had  been  selected  under  each  system,  and  with  each 
of  their  immediate  superiors.  This  was  done  to 
obtain  their  perceptions  on  the  operation  of  the 
system  as  well  as  on  the  performance  of  those 
selected  under  the  system.  For  yet  another  perspec¬ 
tive  on  the  process,  we  administered  written  ques¬ 
tionnaires  to  groups  of  nonsupervisory,  subordinate 
employees,  most  of  whom  work  for  first-line 
supei  visors  we  interviewed.  Over  200  individuals 
participated  in  the  interviews  or  employee  question¬ 
naire  survey. 

In  the  next  section  of  the  report,  we  present  our 
findings  from  these  efforts.  First  we  describe  the 
typical  approach  used  by  most  agencies  in  selecting 
first-line  supervisors  and  offer  our  assessment  of 
how  well  that  approach  is  working.  We  then 
describe  situations  agencies  may  face  which  call  for 
strategies  different  from  the  typical  assessment 
approach.  In  discussing  these  situations,  we 
describe  how  the  alternative  strategies  we  examined 
may  be  useful  in  helping  agencies  to  meet  those 
special  needs.  These  strategics  are  explained  in 
further  detail  in  appendixes  1  through  4.  The 
advantages  and  disadvantages  we  discuss  for  each 
strategy  arc  based  upon  experiences  and  opinions  of 
interviewees  in  the  studied  agencies.  Finally,  we 
present  recommendations  for  action  by  OPM  and  the 
agencies  to  improve  Federal  supervisory  selection 
programs. 
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FINDINGS 


HOW  AGENCIES  TYPICALLY  FILL 
SUPERVISORY  JOBS 


Our  survey  of  Federal  agencies  indicates  that  few 
different  or  innovative  systems  for  selecting  first- 
line  supervisors  exist.  For  the  most  part,  agencies 
select  individuals  for  first-line  supervisory  positions 
using  a  process  identical  to  the  one  they  use  for  all 
other  types  of  jobs.  The  primary  features  of  the 
typical  selection  system  include  the  following: 

■  Job  analysis  is  performed  to 
identify  the  requirements  of  the 
job  being  filled. 

■  An  individual  job  announcement 
is  posted  for  each  job  as  it 
becomes  vacant. 

■  Interested  individuals  must 
submit  an  application  form  for 
each  job  as  it  is  announced. 

■  Generally,  determination  of  the 
best  qualified  applicants  is  based 
upon  an  assessment  of  each 
applicant's  training  and  experi¬ 
ence  as  they  relate  to  the  job 
requirements  and  is  conducted 
wholly  through  review  of  written 
documentation  (c.g.,  applicant's 
qualifications  statement  and 
supervisor’s  appraisal  of  the  indi¬ 
vidual). 

■  When  interviews  are'  used  in  the 
process,  they  are  of  the  selection, 
rather  than  the  evaluation, 
variety.  In  other  words,  only 
those  individuals  who  arc  in  the 


best  qualified  category  arc  inter¬ 
viewed.  Interview  questions 
tend  lobe  general  rather  than 
structured,  and  arc  not  tailored  in 
such  a  way  that  can  be  used  to 
score  the  interviewee  on 
specific  job-related  factors. 

Our  review  of  materials  submitted  by  agencies 
revealed  that  there  is  an  emphasis  in  the  evaluation 
process  on  the  assessment  of  technical,  rather  than 
supervisory,  knowledge,  skills,  and  abilities.  Further 
evidence  of  the  tendency  of  agencies  to  emphasize 
technical  factors  is  provided  by  data  recently 
collected  in  an  MSPB  survey  of  Federal  personnel 
specialists.  When  asked  whether  their  agencies 
emphasized  competence  in  the  technical  area  versus 
competence  in  supervisory  skills  and  abilities  in  the 
selection  of  first-line  supervisors,  42  percent 
reported  that  their  agencies  emphasize  competence 
in  the  technical  area.  Only  10  percent  report  that 
competence  in  supervisory  skills  and  abilities  is 
emphasized  most.  Thirty-seven  percent  report  that 
technical  and  supervisory  competencies  arc  empha¬ 
sized  equally.  (The  remaining  respondents  did  not 
indicate  the  factors  that  arc  emphasized.)  Perhaps 
these  responses  are  not  so  surprising,  in  light  of  our 
previous  discussion  concerning  difficulties  found  in 
evaluating  candidates’  supervisory  competencies 
when  they  have  not  had  the  opportunity  to  demon¬ 
strate  such  abilities  before. 

Nonetheless,  most  agencies  from  which  we  received 
information  indicated  that  some  supervisory  factors 
arc  used  in  the  evaluation  process.  In  fact,  there 
exists  much  similarity  among  the  supervisory  factors 
used  by  agencies  in  their  evaluation  and  selection  of 
first-line  supervisors.  The  following  list  depicts  the 
supervisory  evaluation  factors  most  frequently  cited 
by  agencies  submitting  information: 
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ORAL  COMMUNICATION 
WRITTEN  COMMUNICATION 
ANALYTICAL  ABILITY 

ABIUTY  TO  GENERATE  INNOVATIVE  IDEAS 
AND  SOLUTIONS 

ABILITY  TO  ORGANIZE  WORK 

KNOWLEDGE  OF  RESOURCE  (HUMAN,  FISCAL, 
EQUIPMENT,  SPACE)  MANAGEMENT 
PRINCIPLES 

KNOWLEDGE  OF  SUPERVISORY /MANAGEMENT 
TECHNIQUES 

INTERPERSONAL  RELATIONS  SKILLS 
ABIUIJ  TO  TOLERATE  STRESS 
DECISIONMAKING  ABIUTY 
LEADERSHIP 


talivc  agency  because  its  supervisory  cadre  repre¬ 
sents  a  wide  variety  of  occupations  and  grade  levels, 
and  its  selection  program  includes  all  the  representa¬ 
tive  aspects  of  the  typical  approach  that  most 
agencies  reported  they  use.  We  visited  two  VA  sites 
for  our  study,  a  Medical  Center  and  a  Regional 
Office  and  Insurance  Center. 

As  noted  previously,  one  way  to  assess  the  effective¬ 
ness  of  the  selection  system  is  to  determine  how 
well  supervisors  selected  through  the  system  arc 
performing.  We  looked  at  supervisory  performance 
(and  the  selection  system)  at  the  V A  sites  from  a 
number  of  different  perspectives.  First  of  all,  the 
VA  managers  we  interviewed  expressed  general 
satisfaction  with  the  selection  system  and  believed 
that  good  candidates  were  referred  to  them  for 
selection.  These  managers  also  indicated  that  the 
candidates  selected  had  met  their  performance 
expectations  once  they  got  on  the  job.  Most 
individuals  whom  we  interviewed  felt  that  they  were 
adequately  prepared  when  they  entered  their  first 
supervisory  job.  but  some  said  they  would  have  been 
more  comfortable  if  they  had  had  supervisory 
training  before  being  placed  on  the  job. 


INITIATIVE 

Thus,  not  only  do  most  agencies  tend  to  use  the 
same  basic  approach  to  supervisory  selection,  they 
also  use  many  of  the  same  factors  when  evaluating 
supervisory  potential. 

HOW  WELL  DOES  THE  TYPICAL  SYSTEM 
WORK? 


Onf.  Agency’s  Perceptions 


Given  that  our  initial  findings  showed  most  agencies 
are  using  the  same  type  of  selection  strategy,  we 
decided  to  take  a  closer  look  at  how  that  strategy 
fared  in  a  representative  agency.  We  selected  the 
Department  of  Veterans  Affairs  (VA)  as  a  represen- 


Wc  also  collected  information  from  the  subordinates 
of  supervisors  selected  through  the  system.  For  the 
most  part,  subordinates  were  satisfied  with  the 
performance  of  their  supervisors,  although  they  were 
not  always  as  positive  as  the  supervisor’s  superior 
had  been.  This  discrepancy  between  the  second- 
level  supervisor's  impressions  of  a  selectee’s 
performance  and  those  of  their  subordinates  is  not 
surprising,  given  their  differing  expectations.  (For 
example,  where  second-line  superiors  described 
some  supervisors  as  effective  because  they  were 
“tough,”  “no-nonsense,”  or  “demanding,”  the 
supervisors’  subordinates  did  not  always  universally 
agree  with  that  viewpoint.)  As  yet  another  indicator 
of  supervisory  performance,  we  examined  the 
performance  appraisal  records  of  supervisors  who 
had  been  selected  at  the  two  sites,  and  found  that  all 
ratings  for  the  2  years  after  their  selection  were  at 
least  fully  successful  or  belter. 
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The  findings  from  our  site  visits  at  VA  seem  to 
recommend  the  “typical”  approach  to  selecting 
supervisors.  Nonetheless,  despite  the  positive 
findings  at  the  two  sites  we  visited,  there  are  other 
indicators  which  caution  us  from  drawing  the 
conclusion  that  this  type  of  strategy  is  always 
effective  in  providing  all  agencies  using  this 
approach  with  the  best  supervisors  available.  Some 
of  these  other  indicators  arc  discussed  in  the 
following  sections. 

Indicators  From  Agency  Data 


Even  though  the  managers  we  interviewed  at  the  VA 
sites  arc  generally  satisfied  with  the  pool  of  candi¬ 
dates  presented  to  them  for  selection,  a  number  of 
agencies  responding  to  our  initial  inquiry  indicated 
that  they  need  better  methods  for  identifying 
candidates  for  first-line  supervisory  positions.  This 
suggests  to  us  that  agency  officials  believe  that  there 
arc  greater  numbers  of  higher  quality  individuals 
available  for  selection  than  arc  currently  being 
identified. 

Additionally,  even  though  the  performance  apprais¬ 
als  reviewed  certainly  suggest  that  supervisors 
selected  under  the  typical  system  arc  performing 
well,  we  arc  mindful  that  performance  appraisals 
may  not  be  entirely  accurate  indicators  of  the  quality 
of  supervisory  performance.  In  fact,  a  1988  MSPB 
study  concluded  that  performance  ratings  being 
given  to  Government  employees  arc,  by  definition, 
inflated.4 

Indicators  From  tiie  Si  per  visor  v  Probation 
Program 


A  potentially  useful  indicator  of  how  well  the 
typical  selection  approach  is  working  concerns  the 
supervisory  probation  period.  Data  from  agencies’ 
experiences  with  supervisory  probation  can  serve  as 


4  U.S.  Merit  Systems  Protection  Hoard,  “Toward  Hffcctivc 
Performance  Management  in  the  Federal  Government,”  July  1988. 


one  very  general  measure  of  supervisory  perform¬ 
ance.  All  new  first-line  supervisors  arc  required  to 
serve  a  probation  period  as  the  final  step  in  the  total 
selection  pioccss.  This  requirement  derives  from 
the  Civil  Service  Reform  Act  of  1978  (CSRA) 5  and 
was  imposed  for  two  major  reasons.  First,  those 
concerned  recognized  that  the  skills  and  abilities 
required  of  a  supervisor  arc  unique,  and  that  all  of 
them  arc  not  easily  learned  or  developed  while  in 
nonsupervisory  jobs.  Since  the  best  way  to  assess 
how  well  a  person  will  perform  in  a  job  is  to 
actually  let  him  or  her  do  the  job,  the  new  probation¬ 
ary  period  was  launched  as  a  means  to  permit  an 
agency  to  apply  this  test  period  and  judge  the 
employee’s  performance  before  making  the  supervi¬ 
sory  selection  final. 

Making  personnel  management  more  efficient  and 
less  cumbersome  is  the  second  reason  the  CSRA 
framers  introduced  supervisory  probation.  The 
probationary  period  provides  a  means  to  deal  with 
unsuccessful  performers  without  undue  formality  or 
hardship.  Those  who  cannot  successfully  complete 
the  probationary  period  because  of  deficiencies  in 
supervisory  or  managerial  performance  must  be 
relumed  to  a  nonsupervisory  or  nonmanagerial 
position.  Generally  speaking,  the  agency  would 
place  such  an  employee  in  a  position  which  is  no 
lower  graded  or  lower  paid  than  the  one  the  em¬ 
ployee  left  to  accept  the  supervisory  job.  That 
probation  was  intended  to  be  a  final  lest  of  supervi¬ 
sory  skills  is  underscored  by  the  availability  of  other 
procedures  for  resolving  problems  with  technical 
performance.6 


5  5  i:  s.c  J32i. 

^  In  19X8,  MSPB  upheld  an  agency’s  removal  of  a  new 
supervisor  for  deficiencies  rclaling  lo  his  icchnic.il  ability  lo 
perform  the  substantive  portions  of  his  position,  and  not  his 
managerial  or  supervisory  duties.  In  that  ease,  MSPB  determined 
that  the  agency  was  not  under  an  obligation  lo  return  the  employee 
to  the  grade  from  which  he  had  been  promoted  upon  entering  his 
first  supervisory-  position.  Stewart  vs.  Department  of  the  Air  Force. 
35  M  S  P  R  622(19X8). 
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How  do  first-time  Federal  supervisors  perform  on 
this  “final  test?”  To  date,  no  comprehensive  data 
arc  available  to  make  a  determination  because  OPM 
has  not  evaluated  Govemmeniwide  experience  with 
the  CSRA  requirement.  Furthermore,  OPM  has 
informed  MSPB  that  it  cannot  compute  an  exact, 
Govemmeniwide  failure  rate  for  supervisory 
probationers,  because  data  about  the  number  of  first¬ 
time  supervisors  arc  not  captured  in  OPM’s  Central 
Personnel  Data  File  (CPDF).  However,  because 
personnel  offices  use  a  unique  authority  code  when 
employees  arc  reassigned  or  changed  to  a  lower 
grade  when  they  fail  to  satisfactorily  complete  the 
supervisory  probationary  period,  OPM  can  count 
how  many  individuals  arc  included  in  this  group. 

By  comparing  the  number  of  employees  who  were 
coded  in  the  C PDF  as  supervisor  or  manager  as  of 
September  1987,  but  were  not  similarly  coded  as  of 
September  1986,  OPM  estimated  the  number  of 
employees  who  might  have  been  serving  a  supervi¬ 
sory  probationary  period  during  fiscal  year  1987. 
Based  on  these  data,  they  calculate  a  failure  rate  of 
two-tenths  of  1  percent,  or  2  failures  for  every 
l.OOO.7 

In  light  of  the  problems  OPM  found  in  trying  to 
estimate  a  failure  rate,  this  estimate  is  tenuous,  at 
best.  Therefore,  until  a  more  comprehensive 
examination  of  the  supervisory  probation  program  is 
undertaken  by  OPM,  we  do  not  believe  that  existing 
data  can  substantially  support  the  notion  that  the 
typical  selection  strategy  used  is  operating  effec¬ 
tively. 

Indicators  From  Questionnaire  Surveys 


Another  way  to  measure  the  effectiveness  of  the 
typical  selection  strategy  is  to  look  at  subordinates’ 
perceptions  about  their  supervisors  and  supervisors’ 
perceptions  about  their  own  performance.  A  1986 
MSPB  survey8  provides  some  interesting  insights 


7  Source:  U  S.  Office  of  Personnel  Management,  Personnel 
Systems  and  Oversight  Group,  September  1988. 


into  how  employees  view  their  supervisors.  Data 
from  the  survey  indicate  only  one  out  of  two  (50 
percent)  respondents  agree  that  their  supervisor  has 
good  leadership  qualities.  (This  contrasts  notably 
with  the  nearly  two  out  of  three  (65  percent)  who 
say  that  their  supervisor  has  good  technical  skills.) 
Even  fewer  than  one  out  of  two  employees  (46 
percent)  believe  that  their  supervisor  has  organized 
the  work  group  effectively  to  get  the  work  done. 
Supervisors  do  receive  Letter  marks  for  effective 
two-way  communication  and  fair  treatment  of 
subordinates  (64  percent  and  67  percent  of  employ¬ 
ees,  respectively,  express  these  views  concerning 
their  supervisors).  Nonetheless,  the  perceptions 
evidenced  by  the  survey  portray  a  workplace 
scenario  that  is  quite  different  from  the  situation 
suggested  by  OPM  where  only  2  out  of  1 ,000 
supervisors  perform  less  than  satisfactorily  during 
the  first  year  of  their  supervisory  assignment. 

Another  nationwide  survey  of  more  than  7,000 
private  and  public  sector  supervisors  concluded  that 
the  “***gcncral  suspicion**"  that  most  supervisors 
muddle  through***is  in  part  borne  out.”9  Fully  75 
percent  of  the  supervisors  responding  to  the  survey 
report  that  they  feel  overwhelmed  by  organizational 
communication  and  information  needs  and  complain 
about  the  amount  of  paperwork  they  have  to  handle. 
Less  than  half  consider  themselves  part  of  company 
management.  Significantly,  21  percent  of  the  first- 
level  white-collar  supervisors  arc  unhappy  and 
would  return  to  the  rank-and-file  if  they  could  do  so 
without  a  pay  reduction  or  loss  of  stature. 


8 

The  1986  Merit  Principles  Survey  conducted  by  MSPB  was 

an  extensive  survey  of  21,620  Federal  employees,  designed  to 

provide  valid  results  for  the  entire  full-time,  permanent  Federal 

work  force  regarding  a  variety  of  personnel  management  issues. 

9 

Bittcl,  I.estcr  R.,  and  Jackson  F.  Ramsey,  “The  Limited, 
Traditional  World  of  Supervisors,”  Harvard  Business  Review, 
July-August  1982,  p.  26. 
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Thus,  these  data,  in  addition  to  other  indicators 
discussed  above,  lead  us  to  believe  that  the  selection 
strategy  typically  used  by  most  agencies  (i.c.,  one 
relying  primarily  on  evaluation  of  previous  training 
and  work  experience)  may  not  be  adequate  for 
meeting  selection  needs  in  all  situations.  In  the  next 
section  we  describe  sonu  circumstances  which  call 
for  the  development  and  implementation  of  more 
innovative  approaches  to  evaluation  than  that 
typically  used  by  agencies.  We  envision  that  these 
circumstances  arc  relatively  commonplace  in  many 
agencies,  and  we  describe  how  a  few  agencies  have 
in  fact  developed  alternative  selection  strategies  to 
deal  with  the  needs  arising  from  these  circum¬ 
stances.  One  thing  will  become  apparent  as  we 
describe  these  selection  strategics:  they  all  move 
away  from  inferring  an  applicant’s  skills  or  abilities 
based  upon  prior  training  and  experience,  to  measur¬ 
ing  them  more  directly  through  alternative  methods. 

WHEN  ALTERNATIVE  SELECTION 
STRATEGIES  ARE  NEEDED 


Jobs  Rt:q(  mrsc  a  Hk.ii  Deoree  of  Technical  as 
VVfi.i.  as  Supervisory  Expertise 


Introduction.  The  first-line  supervisor’s  job  gener¬ 
ally  requires  more  emphasis  on  supervisory  skills 
than  technical  skills.  However,  some  first-line 
supervisory  jobs  require  a  greater  degree  of  techni¬ 
cal  expertise  than  the  norm.  That  is,  the  technical 
area  to  be  supervised  is  so  complex  that  the  supervi¬ 
sor  would  need  more  than  just  a  fundamental 
understanding  of  the  technical  aspects  of  the 
subordinates’  jobs  in  order  to  carry  out  the  supervi¬ 
sory  duties  of  the  job  (c.g.,  assigning  work  to 
subordinates  or  evaluating  subordinates’  perform¬ 
ance).  In  such  eases,  deficiencies  in  technical 
knowledge  would  drastically  hinder  the  supervisor 
in  performing  the  supervisory  aspects  or  the  job. 
While  the  typically  used  method  may  be  sufficient 
for  assessing  this  technical  knowledge,  what  is 
needed  is  a  system  that  can  efficiently  assess  both 
supervisory  and  technical  factors  at  the  same  time. 


Written  Knowledge  Tests.  Written  knowledge 
tests  such  as  the  one  developed  and  administered  by 
the  Department  of  Justice’s  U.S.  Marshals  Service 
(USMS),  provide  an  opportunity  to  objectively 
measure  candidates’  knowledge  concerning  both 
technical  and  supervisory  aspects  of  the  job.  The 
knowledge  areas  measured  by  the  Marshals  Service 
test  were  identified  through  job  analysis  as  critical  to 
performance  of  Supervisory  Deputy  U.S.  Marshal 
jobs.  The  development  of  the  test  involved  exten¬ 
sive  input  from  individuals  very  knowledgeable  of 
work  requirements  of  the  jobs  being  filled,  as  well 
as  the  specific  kinds  of  knowlcdf  i  required  to  carry 
out  the  duties  of  those  jobs.  In  a  law  enforcement 
environment  such  as  the  Marshals  Service,  it  is 
critical  for  first-line  supervisors  to  have  a  thorough 
understanding  of  the  myriad  of  laws  and  regulations 
governing  'he  work  performed  by  the  work  unit. 
Often,  these  supervisors  must  quickly  provide  advice 
and  direction  requiring  knowledge  of  these  laws  and 
regulations,  and  mistakes  made  by  not  following  the 
laws  and  regulations  can  potentially  result  in  loss  of 
life. 

In  addition  to  providing  direct,  objective  assess¬ 
ments  of  applicants’  technical  as  well  as  supervisory 
knowledge,  written  knowledge  tests  also  have  other 
advantages  for  agencies.  For  example,  they  arc 
potentially  easier  to  score  than  typical  procedures 
used  to  evaluate  documentation  concerning  appli¬ 
cants’  training  and  experience;  applicants  perceive 
these  measures  as  fairer  than  subjective  assessments, 
since  all  applicants  arc  scored  the  same  way;  and 
agencies  can  use  data  gathered  from  repeated 
administrations  of  these  tests  to  determine  their 
future  training  and  development  needs. 

As  with  any  method,  however,  there  arc  some 
disadvantages  to  using  written  tests.  Some  appli¬ 
cants  arc  simply  not  “test-lakers,”  and  perform 
poorly  on  written  tests  because  of  factors  unrelated 
to  their  possession  of  the  knowledge  in  question.10 


^Guion,  Robert  M.,  “Personnel  resting,”  MeGraw  Hill, 
1965,  p.  35. 
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Also,  development  and  administration  of  such  tests 
can  be  expensive  (although  costs  incurred  for  “bad 
selections”  of  supervisors  may  be  much  greater  in 
the  long  run).  Finally,  ascertaining  that  an  applicant 
possesses  a  certain  level  of  knowledge  docs  not 
ensure  that  the  applicant  will  apply  that  knowledge, 
once  in  the  job.  By  the  same  token,  however,  even 
the  traditionally  used  method  cannot  ensure  that 
prior  exposure  to  experiences  requiring  certain  kinds 
of  knowledge  will  result  in  the  application  of  those 
kinds  of  knowledge  in  the  new  position. 

Prenlacement  T raining.  Another  possible  strategy 
to  help  ensure  that  supervisors  possess  a  certain 
level  of  technical  and  supervisory  proficiency 
involves  structured  training  prior  to  placement  in  the 
job.  One  agency  using  this  approach  is  the  Depart¬ 
ment  of  Labor’s  Mine  Safety  and  Health  Admini¬ 
stration  (MSHA).  The  strategy  developed  by  MSHA 
requires  all  those  selected  as  candidates  for  Supervi¬ 
sory  Coal  Mine  Inspector  positions  to  undergo  an 
intensive  1-ycar  training  program  prior  to  placement 
in  the  supervisory  job.  The  approach  MSHA  uses 
includes  training  in  specific  technical  topics  as  well 
as  training  in  supervisory  skills  and  subjects.  The 
program  uses  centrally  administered,  formal 
classroom  sessions  as  well  as  short-term  job  assign¬ 
ments  in  which  participants  work  with  an  incumbent 
first-line  supervisor.  (Near  the  end  of  their  training 
program,  the  candidates  may  also  fill  in  as  acting 
supervisors  when  a  permanent  supervisor  is  away 
from  the  office  at  training  or  for  other  reasons.) 
Throughout  the  course  of  the  program,  participants 
arc  formally  tested  to  ensure  they  are  acquiring  the 
desired  level  of  knowledge  and  skills. 

Because  such  a  program  can  be  relatively  cosily,"  it 
obviously  would  not  be  practical  for  all  supervisory 


1 1  MSHA  is  fortunate  in  being  able  to  instruct  its  supervisory 
candidates  in  its  own  fully  staffed  training  facility,  the  National 
Mine  Health  and  Safety  Academy,  in  Bccklcy,  West  Virginia.  The 
Academy  primarily  offers  courses  in  various  aspects  of  mine 
safety  and  health  for  MSIlA’s  inspectors  and  for  individuals  from 
state  and  local  government  agencies,  mine  management,  and  labor 
organizations. 


selection  situations.  Even  when  the  applicant  pool 
is  reduced  by  having  employees  compete  for  these 
“training”  positions,  a  significant  commitment  of 
resources  is  still  required  to  administer  this  type  of 
training.  However,  if  mistakes  made  by  supervisors 
due  to  lack  of  knowledge  or  skill  in  the  technical 
area  arc  potentially  very  costly,  the  investment  in 
such  a  training  program  might  prove  worthwhile. 

For  example,  MSHA  Supervisory  Coal  Mine 
Inspectors  must  perform  the  final  technical  review 
for  adequacy  of  ventilation  plans  for  coal  mines.  If 
a  deficient  plan  is  approved  because  of  a  lack  of 
technical  expertise,  the  result  could  be  a  mine 
explosion  or  other  disaster. 

A  Iso,  if  it  is  critical  that  either  technical  or  supervi¬ 
sory  knowledge  be  applied  in  a  standard  manner,  a 
centralized  training  approach  such  as  MSHA’s 
program  may  prove  very  useful.  Centralized 
classroom  training  with  subsequent  testing  of 
participants’  comprehension  of  the  material  ensures 
that  all  participants  possess  an  acceptable  level  of 
knowledge  about  the  subject  in  question.  Further¬ 
more,  the  training  can  help  ensure  that  these  future 
supervisors  consistently  apply  the  knowledge  and 
skills,  according  to  established  standards.  This  is 
critical  for  an  organization  like  MSHA,  which  has 
jurisdiction  across  the  Nation  through  a  network  of 
districts,  subdistricts,  and  field  offices.  Because  a 
single  mining  company  may  have  mines  located  in 
different  States,  if  a  supervisor’s  interpretation  of 
the  law  in  a  company  mine  in  one  State  conflicts 
with  another  supervisor’s  interpretation  in  a  different 
Stale,  serious  questions  could  arise  about  the 
consistency  of  MSHA’s  regulation  of  mining 
activities. 

Jobs  in  SiRKSsm.  Environments 


Introduction.  Because  of  the  nature  of  the  techni¬ 
cal  work  performed  in  their  units,  some  First-line 
supervisors  work  in  especially  stressful  environ¬ 
ments.  The  stress  may  be  caused  by  such  factors  as 
the  intensity  or  pace  of  the  technical  work,  or  the 
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need  for  split-second  decisions  which  may  have 
safety  (even  life  and  death)  implications.  In  these 
situations,  one  of  the  supervisor’s  most  important 
functions  is  to  minimize  conflicts  in  order  to  keep 
the  work  flowing  smoothly.  Therefore,  it  is  critical 
that  the  supervisor  be  especially  adept  at  interper¬ 
sonal  skills  and  abilities  in  order  to  alleviate  tensions 
among  members  of  the  group  that  might  impede  the 
group’s  performance.  However,  these  interpersonal 
skills  and  abilities  arc  notoriously  difficult  to 
measure  reliably.  By  their  very  nature,  interpersonal 
skills  involve  situations  with  both  a  sender  and  a 
recipient,  and  the  effectiveness  of  their  interaction 
depends  on  the  recipient’s  perception  of  that 
interaction.  In  a  sense,  one  cannot  use  an  objective 
yardstick  to  measure  these  skills  since  they  are  truly 
subjective  and  really  exist  in  the  “eye  of  the 
beholder.”  Thus,  trying  to  infer  the  effectiveness  of 
an  applicant’s  ability  to  interact  effectively  strictly 
from  previous  training  and  work  experience,  as  is 
done  with  the  traditionally  used  approach  for 
assessing  supervisory  applicants,  is  often  not 
successful. 

Peer  Rating  Strategy.  In  an  attempt  to  more 
directly  assess  supervisory  skills  and  abilities,  the 
Department  of  Transportation’s  Federal  Aviation 
Administration  (FAA)  has  adopted  a  strategy 
designed  to  evaluate  applicants  for  certain  supervi¬ 
sory  jobs  from  the  perspective  of  the  subordinate. 
With  this  approach,  applicants  for  FAA’s  Supervi¬ 
sory  Air  Traffic  Controller  jobs  arc  given  peer 
ratings  (i.c.,  ratings  made  by  other  Air  Traffic 
Controllers  with  whom  they  work)  on  supervisory 
skill  and  ability.  The  elements  rated  include  (but  arc 
not  limited  to)  interpersonal  and  communication 
skills  and  abilities.  Applicants  are  shown  only  their 
composite  scores  and  do  not  learn  how  individual 
peers  have  rated  them.  Since  this  strategy  is  based 
on  the  perceptions  of  those  who  have  worked  closely 
with  the  applicants  under  conditions  widely  recog¬ 
nized  as  stressful  (i.c.,  air  traffic  control),  users 
whom  we  surveyed  agree  that  it  can  provide  unique 
and  useful  insights  concerning  applicants’  potential 


to  perform  the  supervisory  job.  Because  the 
interpersonal  skills  involve  interaction  with  others,  a 
strategy  which  elicits  opinions  directly  from  those 
who  have  the  greatest  opportunity  to  interact  with 
the  applicants  provides  an  effective  approach  for 
predicting  future  performance  in  a  job  requiring 
proficiency  in  interpersonal  skills  and  abilities. 

However,  a  potential  disadvantage  of  a  strategy 
using  peer  ratings  is  that  the  work  force  might 
perceive  the  selection  process  as  reduced  to  little 
more  than  a  “popularity  contest.”  This  perception 
may  result  in  distrust  of  the  system  by  employees 
and  concern  by  them  over  the  capabilities  of 
supervisors  who  arc  selected.  However,  the  use  of 
peer  rating  forms  which  are  based  on  very  specific, 
job-related  dimensions  and  which  require  ratings 
that  arc  supported  by  examples  of  observed  behavior 
can  help  ensure  selections  are  based  on  applicants’ 
actual  performance  rather  than  on  general  impres¬ 
sions  of  popularity.  Also,  by  randomly  selecting 
peers  to  make  the  ratings  (or  using  some  combina¬ 
tion  of  random  assignment  and  personal  selection  by 
the  candidate),  the  ratings  can  be  made  more 
accurate  and  job-related. 

Jobs  in  Organizations  Which  Emphasize 
Employee  Participation 


Introduction.  Many  organizations  appear  to  be 
increasingly  interested  in  fostering  a  climate  of 
participative  management.  Organizations  may  try  to 
involve  employees  more  in  decisionmaking  to 
improve  employee-management  relations,  increase 
employee  commitment  and  identification  with  the 
organization’s  goals,  enhance  the  quality  of  organ¬ 
izational  output  (goods  or  services),  or  achieve  other 
objectives.12  Whatever  the  reasons  for  promoting 

12  In  February  1986,  President  Reagan  signed  F.xccutivc 
Order  1 2552,  to  establish  a  comprehensive  program  for  the 
improvement  of  productivity  throughout  all  executive  departments 
and  agencies.  An  important  aspect  of  that  program,  Total  Quality 
Management  (TQM),  is  encouragement  of  employee  participation 
in  the  productivity  improvement  effort. 
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participative  management,  decisions  concerning 
supervisory  selections  can  become  a  primary  focus 
for  employee  participation. 

Peer  Rating  Strategy.  The  typical  procedure  for 
assessing  candidates  for  supervisory  positions  offers 
little  or  no  opportunity  for  subordinates  to  have  a 
voice  in  the  decision  made.  However,  a  strategy 
using  peer  ratings,  such  as  the  one  used  by  FAA  and 
previously  described  in  the  section  on  jobs  in 
stressful  environments,  can  give  employees  this 
opportunity.  When  employees  are  able  to  partici¬ 
pate  by  providing  input  concerning  qualifications  of 
applicants  for  supervisory  jobs,  they  are  more  likely 
to  accept  selections  subsequently  made.  Fostering 
employee  participation  through  peer  ratings  also 
helps  communicate  to  employees  that  their  input  is 
important  to  management,  a  result  that  can  enhance 
the  working  environment 

As  discussed  previously,  there  arc  potential  disad¬ 
vantages  to  the  use  of  peer  ratings.  This  strategy 
can  actually  hinder  an  organization’s  attempt  to 
enhance  its  working  climate  if  peer  ratings  arc 
elicited  in  the  selection  process  but  aren’t  used  in 
making  the  final  decisions  (c.g.,  a  candidate  who 
was  consistently  rated  poorly  by  peers  is  nonetheless 
selected  for  a  supervisory  job).  On  the  other  hand, 
when  employees  see,  over  time,  that  their  input  is 
seriously  considered  by  selecting  officials,  the 
organization  stands  to  benefit  in  the  ways  discussed 
above. 

Jobs  in  Organizations  Anticipating  Major 
Turnover  in  Supervisory  Ranks 


Introduction.  Occasionally,  an  organization  finds 
itself  anticipating  a  major  upheaval  in  the  work 
force.  This  change  may  be  due  to  modifications  in 
the  work  performed  (e.g.,  technological  advances 
require  the  need  for  a  new  mix  of  skill  or  knowledge 
in  employees),  or  cyclical  changes  in  work  force 
demographics  (e.g.,  a  large  percentage  of  supervi¬ 
sors  arc  nearing  retirement  age).  In  such  cases,  the 


organization  may  have  to  intensify  both  its  hiring 
efforts  and  its  training  efforts.  Under  normal 
conditions,  first-line  supervisors  in  most  Federal 
agencies  receive  training  after  being  placed  in  their 
new  jobs.  Sometimes  this  is  not  done  in  a  timely 
manner  and  the  new  supervisors  can  wait  as  long  as 
1  year  after  placement  before  being  trained.  If  a 
significant  turnover  is  expected  among  current 
supervisors,  a  sudden  dramatic  increase  in  the 
number  of  new  first-line  supervisors  can  add 
significantly  to  an  agency’s  supervisory  training 
requirements  and  delay  training  even  further.  In 
turn,  hiring  or  promoting  a  large  number  of  first-line 
supervisors  who  will  remain  untrained  for  signifi¬ 
cant  periods  can  be  very  disruptive  and  costly. 

Costs  may  involve: 

■  Loss  in  group  productivity  because  the 
new  supervisors  have  not  developed 
skills  for  assigning  and  prioritizing 
subordinates’  work; 

■  Loss  in  group  productivity  due  to  frustra¬ 
tion  on  the  part  of  subordinates  when 
the  new  supervisors  have  not  learned  to 
handle  the  “people”  part  of  the  job; 

■  Increased  grievances  resulting  from 
inappropriate  administrative  actions  taken 
by  new  supervisors  who  are  untrained  in 
agency  policies  and  procedures;  and 

■  Increased  turnover  (of  subordinates  and 
supervisors)  frustrated  with  the  trial- 
and-error  learning  process  that  occurs  in 
the  absence  of  formal  supervisory 
training. 

Preplacement  Training.  In  contrast  to  the  typical 
selection  program  with  its  lack  of  supervisory 
training  prior  to  placement,  is  the  Mine  Safety  and 
Health  Administration’s  program  with  its  strategy  of 
using  preplacement  supervisory  training.  Such  a 
strategy-  which  we  discussed  previously  in  relation 


A  REPORT  BY  THE  U.S.  MERIT  SYSTEMS  PROTECTION  BOARD 


FIRST-LINE  SUPERVISORY  SELECTION  IN  THE  FEDERAL  GOVERNMENT 


12 


to  jobs  having  extraordinary  tech-;"d  require¬ 
ments-  can  provide  an  efficient,  cost  effective 
alternative  for  post-selection  supervisory  training, 
especially  when  a  large  turnover  is  anticipated. 

When  MSHA  was  faced  with  a  sizable  turnover 
among  its  first-line  supervisors,  top  management 
recognized  that  it  needed  a  special  effort  to  ensure  a 
smooth  transition  of  large  numbers  of  employees 
into  supervisory  jobs.  To  spare  itself  some  of  the 
potential  costs  previously  described,  MSHA  decided 
to  provide  training  in  critical  supervisory  skills  (as 
well  as  technical  knowledge)  prior  to  placement  on 
the  job.  In  this  way,  the  organization’s  new  supervi¬ 
sors  could  “hit  the  ground  running,”  making  this 
significant  turnover  in  their  supervisory  ranks  as 
trouble-free  as  possible.  While  such  an  intensive 
training  effort  might  not  be  economically  feasible  in 
all  situations,  in  an  organization  trying  to  plan  for 
significant  changes  in  the  work  force,  such  a 
strategy  can  prove  very  prudent. 

Jobs  in  Organizations  with  a  Large  or 
Geographically  Dispersed  Work  Force 


Introduction.  The  typical  selection  procedure  uses 
job  analysis  to  identify  relevant  skills  or  knowledge 
for  individual  supervisory  jobs  and  announces 
vacancies  for  supervisory  positions  as  they  occur. 
Candidates  interested  in  these  positions  must  submit 
a  separate  application  package  (i.e.,  documentation 
concerning  qualifications)  for  each  position.  This 
strategy  is  probably  an  adequate  one  when  the  work 
force  is  small  enough  in  numbers  and  geographic 
dispersion  is  minimal  enough  to  ensure  that  all 
interested  employees  Icam  of  job  vacancies  and 
have  adequate  opportunities  to  apply.  However, 
many  agencies  are  so  large  or  geographically 
dispersed  that  vacancies  are  not  communicated  to  all 
interested  employees  in  a  reliable  and  timely 
manner.  And,  in  these  agencies,  many  employees 
icam  of  the  vacancies  too  late  to  apply.  Addition¬ 
ally,  some  agencies  may  have  many  similar  jobs  in 
one  occupation.  In  these  cases,  applicants  may 


become  frustrated  at  having  to  submit  numerous 
application  packages  for  vacancies  requiring  the 
same  information.  Also,  many  resources  may  be 
expended  to  conduct  repetitive,  separate  job  analy¬ 
ses  for  positions  that  are,  in  fact,  very  similar. 

Centralized.  Automated  Referral  System.  A  more 
efficient  strategy  for  such  situations  is  one  which 
establishes  standard  evaluation  criteria  for  related 
jobs  (as  determined  through  large-scale  job  analy¬ 
ses),  and  can  be  administered  through  a  centralized, 
automated  system.  Such  an  approach  could  provide 
sound  assessment  of  applicants’  capabilities  while 
proving  efficient  to  operate.  The  Department  of  the 
Army  has  developed  and  implemented  such  a 
procedure,  the  Army  Civilian  Career  Evaluation 
System  (ACCES).  ACCES  is  used  to  fill  supervi¬ 
sory  (and  nonsupervisory)  positions  in  many 
occupations  at  mid-level  grades  and  above. 

Through  extensive  job  analyses,  the  Department  of 
the  Army  identified  inventories  of  evaluation  criteria 
(knowledge,  skills,  or  abilities-KSA’s)  relevant  to 
groups  of  related  jobs  (a  group  of  related  jobs  is 
referred  to  as  a  “career  program”  in  ACCES).  An 
employee  interested  in  being  considered  for  job 
referrals  under  ACCES  only  needs  to  submit  one 
application  package.  This  package  contains  descrip¬ 
tions  of  accomplishments,  and  self  and  supervisory 
ratings  on  evaluation  criteria  relevant  to  a  specific 
career  program.  This  information  is  stored  in  a 
central  computer,  and  whenever  a  vacancy  occurs  of 
the  type  and  location  in  which  the  applicant  is 
interested,  the  applicant  is  automatically  considered 
for  the  position. 

An  applicant  does  not  receive  an  overall  referral 
score  or  ranking  after  submitting  an  application 
package  to  ACCES,  because  a  candidate’s  referral 
score  and  ranking  may  change  with  every  vacancy 
filled.  This  is  due  to  two  aspects  of  the  system: 

1)  the  KSA's  used  in  the  evaluation  process  arc 
weighted  according  to  the  particular  requirements  of 
each  position  applied  for  (and  the  applicant’s 
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referral  score  is  based  on  both  the  weights  of  the 
rating  elements  and  the  applicant’s  ratings  on  the 
elements),  and;  2)  the  mix  of  applicants  interested  in 
positions  will  vary  by  location  of  the  vacancies. 

This  type  of  strategy  can  be  used  to  fill  both 
supervisory  and  nonsupervisory  jobs,  but  is  espe¬ 
cially  effective  for  supervisory  jobs.  Supervisory 
jobs  may  vary  greatly  in  terms  of  the  relative 
importance  of  supervisory  abilities  versus  technical 
knowledge,  yet  share  numerous  elements  which  are 
important  as  evaluation  criteria.  Therefore,  an 
agency  filling  many  supervisory  positions  would 
desire  a  selection  strategy  which  systematically 
identifies  important  elements  common  to  many 
supervisory  jobs  and  allows  tailoring  evaluation 
criteria  to  the  job  being  filled.  A  centralized, 
automated  evaluation  and  referral  system  is  particu¬ 
larly  useful  because  of  the  case  with  which  invento¬ 
ries  of  job-relevant  elements  can  be  weighted  to 
reflect  the  needs  of  the  position  being  filled.  If 
applicants  submit  ratings  on  an  entire  inventory  of 
elements  relevant  to  numerous  supervisory  jobs, 
(c.g.,  ratings  which  ACCES  applicants  submit  for  a 
career  program),  they  can  be  considered  automati¬ 
cally  for  any  position  in  which  they  are  interested, 
regardless  of  the  particular  elements  identified  as 
important  for  that  job. 

A  centralized,  automated  referral  system  should 
enable  an  organization  with  a  large,  geographically 
dispersed  work  force  to  fill  jobs  more  quickly  and 
efficiently  (c.g.,  with  ACCES,  referral  lists  are 
routinely  processed  in  a  single  day).  This  type  of 
strategy  also  offers  positive  benefits  for  applicants. 
Not  only  are  they  spared  the  work  of  having  to 
submit  new  application  packages  with  each  vacancy 
(as  they  often  must  do  with  the  traditional  strategy), 
but  they  are  automatically  considered  for  jobs  about 
which  they  otherwise  might  have  been  unaware. 

As  with  any  strategy,  however,  there  are  potential 
problems.  As  noted  previously,  an  applicant’s 
referral  score  may  change  with  every  position.  This 


can  result  in  misunderstanding  when  an  applicant  is 
referred  for  one  job,  but  not  for  another  job  even 
though  both  jobs  may  appear  to  the  applicant  to  be 
identical.  Failure  of  employees  to  understand  how 
the  system  computes  the  referral  scores  each  time 
can  quickly  lead  to  distrust  of  the  entire  evaluation 
and  selection  process.  This  distrust  can  only  be 
alleviated  through  an  extensive  education  of  the 
work  force  concerning  the  operation  of  the  referral 
system. 

Jobs  in  Organizations  Desiring  to  Improve  the 
Representation  of  Women  and  Minorities  in  the 
Supervisory  Ranks 


Introduction.  The  traditionally  used  strategy  for 
selecting  first-line  supervisors  relics  heavily  on  the 
evaluation  of  previous  training  and  work  experi¬ 
ences  of  applicants.  However,  because  many 
occupational  fields  have  been  historically  filled  by 
white  males,  women  and  minorities  by  and  large 
have  not  had  the  opportunities  to  acquire  the  training 
and  experience  deemed  as  desirable  for  supervisory 
positions  which  arc  generally  at  the  higher  grades  in 
the  occupation.  Therefore,  when  a  selection  strategy 
uses  previous  work  experience  and  training  as  a 
basis  for  evaluating  skill  or  ability,  women  and 
minorities  who  have  not  had  the  same  opportunities 
often  find  themselves  at  a  great  disadvantage  (even 
though  they  may  actually  possess  the  skills  needed 
for  the  jobs  they  seek).  When  the  potential  for  this 
situation  exists,  a  more  appropriate  selection 
strategy  is  one  that  doesn’t  rely  on  previous  work 
experience  and  training  but  instead  bases  the 
evaluation  of  applicants’  qualifications  on  actual 
demonstrations  of  present  performance. 

Simulation  Exercises  for  Evaluating 
Performance.  One  strategy  which  evaluates 
present  performance  as  opposed  to  past  training  and 
experience  is  the  use  of  simulation  exercises  in  the 
evaluation  process.  Although  not  specifically 
developed  to  increase  the  representation  of  certain 
groups  in  the  supervisory  work  force,  simulation 
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exercises  are  used  by  both  the  FAA  and  the  USMS 
to  select  supervisors.  With  these  simulation  exer¬ 
cises,  applicants  are  presented  with  scenarios 
depicting  situations  typically  encountered  in  the 
supervisory  job.  Applicants  must  “size  up”  the 
situations  presented,  articulate  the  issues  or  prob¬ 
lems  involved,  and  take  whatever  actions  they  feel 
the  situations  call  for. 

The  simulation  exercises  are  designed  to  enable 
applicants  to  demonstrate  performance  on  supervi¬ 
sory  abilities  such  as  oral  communication,  decision¬ 
making,  and  leadership.  They  rely  on  candidates’ 
“on-the-spot  performance”  relevant  to  these 
abilities.  Therefore,  candidates  aren’t  penalized  in 
the  evaluation  process  by  a  lack  of  prior  opportuni¬ 
ties  to  demonstrate  their  qualifications  through  work 
experience  or  training.  This  enables  women  and 
minority  candidates  to  be  evaluated  along  with 
nonminority  male  candidates  more  fairly  and 
effectively,  based  upon  present  performance. 

Simulation  exercises  have  been  used  in  different 
ways  to  enhance  the  evaluation  process.  For 
example,  while  the  FAA  uses  simulation  exercises 
as  part  of  a  skill-based  interview,  the  USMS  uses 
simulation  exercises  in  an  assessment  center  to 
select  first-line  supervisors.  An  assessment  center  is 
a  method  (not  a  place)  that  typically  uses  numerous 
simulation  exercises  to  evaluate  knowledge,  skills  or 
abilities  in  a  variety  of  performance  situations. 
Because  it  uses  several  simulation  exercises,  an 
assessment  center  can  provide  a  more  complete 
picture  of  candidates’  supervisory  skills  than  may  be 
possible  with  other  approaches. 

The  USMS  assessment  center  consists  of  four 
different  simulation  exercises  and  a  face-to-face 
interview.  Because  any  candidate’s  performance 
can  vary  according  to  the  situation  presented,  the 
opportunity  for  multiple  observations  of  a  candi¬ 
date’s  performance  increases  the  likelihood  that  a 
truer  picture  of  the  candidate  will  emerge.  For 


example,  a  candidate  may  be  able  to  prepare  and 
deliver  a  speech  with  great  skill  in  one  exercise,  but 
have  difficulty  in  giving  performance  feedback  to  a 
problem  employee  in  another  exercise.  The  oppor¬ 
tunity  to  witness  the  candidate’s  “oral  communica¬ 
tion  skill”  in  both  of  these  situations  should  provide 
a  more  complete  perspective  on  the  applicant’s 
potential  than  either  of  these  two  simulation  exer¬ 
cises  could  provide  alone. 

There  are  numerous  advantages  possible  with  a 
selection  strategy  which  uses  simulation  exercises, 
regardless  of  whether  multiple  exercises  are  in¬ 
cluded  in  an  assessment  center  or  exercises  are  used 
individually.  In  addition  to  alleviating  biases 
associated  with  lack  of  opportunity  to  gain  particular 
work  and  training  experiences,  simulation  exercises 
can  provide  practical  information  to  applicants  for 
their  own  developmental  purposes.  For  example, 
both  the  USMS  and  the  FAA  provide  applicants  with 
detailed  feedback  concerning  their  performance  in 
the  simulation  exercises.  This  information  empha¬ 
sizes  both  strengths  and  weaknesses  relative  to  the 
job-related  factors  evaluated  in  the  exercises,  and 
provides  a  firm  basis  on  which  applicants  may 
pursue  activities  to  improve  in  weak  areas.  Such 
feedback  is  rarely  provided  through  traditional 
methods  of  evaluation  for  supervisory  selection.  In 
fact,  with  traditional  methods,  candidates  often 
never  even  learn  of  selection  decisions  made,  much 
less  are  provided  information  concerning  why  they 
were  not  selected.  Such  information  could  be  useful 
for  further  development. 

The  primary  disadvantage  to  the  use  of  simulation 
exercises  which  has  precluded  many  agencies  from 
pursuing  this  approach  is  cost.  The  resource 
commitment  required  to  develop  and  administer  a 
strategy  using  simulation  exercises  can  be  substan¬ 
tial,  at  least  initially.  In  order  to  provide  reliable 
and  valid  assessments  of  applicants’  potential  for 
supervisory  jobs,  the  exercises  must  be  realistic  to 
both  the  applicants  and  those  making  the  evalu¬ 
ations.  The  development  of  realistic  exercises 
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usually  requires  extensive  involvement  of  subject 
matter  experts  (e.g.,  employees  in  jobs  similar  to 
those  being  filled,  who  are  intimately  familiar  with 
the  requirements  of  the  jobs).  Likewise,  the 
administration  of  simulation  exercises  requires 
involvement  of  individuals  well  trained  in  the 
specific  procedures  used  to  evaluate  candidates  in 
these  exercises.  Since  it  is  usually  most  efficient  for 
organizations  to  train  evaluators  and  administer 
simulation  exercises  in  a  central  location,  involve¬ 
ment  of  numerous  employees  in  the  development 
and  operation  of  simulation  exercises  can  incur 
significant  travel  and  lodging  costs,  in  addition  to 
costs  associated  with  the  employees*  absence  from 
their  regular  jobs. 

Nonetheless,  applicants  and  evaluators  involved  in 
the  use  of  simulation  exercises  (at  least  at  FAA  and 
the  USMS)  generally  believe  that  this  strategy 
represents  a  worthwhile  investment  of  resources. 
Most  candidates  see  the  process  as  more  objective 
and  fairer  than  traditional  evaluation  methods,  and 
useful  for  providing  feedback  critical  to  their  further 
development.  Evaluators  perceive  the  exercises  as 
an  effective  vehicle  for  candidates  to  demonstrate 
their  potential  for  supervisory  positions.  Addition¬ 
ally,  most  second  level  supervisors  and  managers  we 
spoke  with  believe  such  exercises  provide  more 
accurate  measures  of  supervisory  ability  than 
traditional  strategies  which  rely  on  inferences 
concerning  previous  training  and  experience. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


Our  review  of  supervisory  selection  practices  among 
Federal  agencies  indicates  that,  by  and  large,  most 
agencies  use  the  same  general  approach  for  evaluat¬ 
ing  candidates  for  first-line  supervisory  positions, 
and  this  approach  is  the  same  one  they  use  to  fill 
their  nonsupervisory  positions.  Information  gath¬ 
ered  from  officials  at  an  agency  representative  of 
other  agencies  using  this  approach,  as  well  as  from 
onsite  interviews  at  that  agency,  indicate  that  the 
approach  seems  to  be  working  satisfactorily. 
Nonetheless,  other  indicators  (e.g.,  comments  from 
agencies  responding  to  our  survey  and  anecdotal 
evidence,  and  results  of  the  earlier  attitude  surveys- 
the  MSPB  Merit  Principles  Survey  and  the  nation¬ 
wide  survey  of  supervisors-discussed  in  a  previous 
section  of  this  report)  strongly  suggest  that  not 
everyone  is  completely  satisfied  with  selection 
strategies  in  use  and  that  certain  situations  may  exist 
which  call  for  other  strategies. 

Because  of  the  diversity  throughout  the  Federal 
sector  in  the  work  performed,  organizational 
environments,  and  agency  missions,  it  is  highly 
unlikely  that  any  one  strategy  for  selecting  first-line 
supervisors  will  be  totally  effective  in  meeting  all 
needs.  We  have  briefly  discussed  some  of  the 
circumstances  which  call  for  strategies  other  than 
those  relying  on  documentation  of  previous  training 
and  experience.  Certainly  one  may  envision 
situations  other  than  those  we  have  described  which 
also  call  for  innovative  selection  strategies.  How¬ 
ever,  we  chose  to  present  in  this  report  the  situations 
likely  to  be  most  prevalent. 

Given  some  of  the  special  selection  situations  we 
identified  in  this  study,  we  strongly  urge  agencies  to 
take  a  much  closer  look  at  their  own  strategies  and 
determine  if  they  are  adequate  for  meeting  their 
individual  selection  needs.  We  believe  that  many 
agencies  fail  to  recognize  the  critical  role  of  first- 
line  supervisors  in  the  organization’s  fulfillment  of 


its  mission,  and  consequently  do  not  devote  the 
efforts  warranted  to  appropriately  fill  these  key  jobs. 
Agencies  that  are  experiencing  organizational 
problems  which  may  be  related  to  the  quality  of 
their  supervisors  may  especially  wish  to  scrutinize 
their  selection  devices  and  consider  alternatives. 
Finally,  demographic  projections  which  portend 
intensified  competition  for  well-qualified  workers13 
make  it  all  the  more  important  that  agencies  select 
carefully  and  effectively  from  among  a  shrinking 
pool  of  candidates. 

Some  of  the  alternative  strategies  discussed  in  this 
report  may  assist  agencies  in  improving  supervisory 
selection  efforts.  In  evaluating  the  adequacy  of  their 
supervisory  selection  systems,  agencies  should  also 
be  aware  of  some  common  factors  we  identified  in 
the  innovative  systems  we  studied.  Notwithstanding 
differences  in  approach,  the  following  key  factors 
appear  to  be  common  to  the  innovative  systems  we 
studied.  Agencies  can  use  these  factors  as  a  frame¬ 
work  for  closer  examination  of  their  existing 
supervisory  selection  systems. 

■  Top  management  is  visibly  supportive  of 
the  system.  In  several  programs,  the 
highest  levels  of  management  clearly  play 
significant  roles  in  the  design  and 
implementation  of  innovative  supervisory 
selection  systems.  They  are  involved 
in  approving  the  conceptual  approaches 
and  most  importantly,  they  view  the 
systems  as  an  integral  part  of  their  human 
resource  management  plan. 


13  A  June  1988  Hudson  Institute  report,  "Civil  Service 
2000,"  highlights  important  demographic  issues  facing  the  Federal 
Government  in  the  1990's. 
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■  The  system  meets  organizational  needs. 
Contrary  to  some  approaches  which 
may  be  driven  by  administrative  or 
procedural  requirements  of  the  personnel 
system,  these  systems  leave  the  distinct 
impression  that,  while  they  exist  in  the 
context  of  the  personnel  system,  the 
program  is  created  fcy  managers  to  meet 
managers’  needs.  This  is  not  to  imply 
that  personnel  office  representatives  play 
insignificant  roles  in  these  situations,  only 
that  they  are  involved  in  a  much 

more  active  partnership  with  management 
than  is  generally  the  case  in  the 
design  and  operation  of  personnel 
programs. 

■  The  system  is  dynamic.  Although  the 
systems  studied  were  in  different  stages 
of  implementation  (some  had  just  begun, 
others  had  been  in  operation  for 
years),  developers  and  administrators  of 
each  keep  a  close  watch  on  the 
changing  needs  of  those  served  by  the 
systems,  and  modify  the  systems 
accordingly.  Several  have  implemented 
formal  procedures  for  obtaining 
feedback  concerning  operation  of  the 
systems.  This  feedback  is  used  as  an 
integral  part  of  ongoing  attempts  to 
improve  the  systems. 

■  The  system  uses  a  sound  measurement 
approach.  The  systems  we  studied 
acknowledge  the  criticality  of  the 
personnel  management  pan  of  a  supervi¬ 
sor’s  job  and  have  developed  ways  to 
assess  the  requisite  skills  or  potential  in 
candidates.  The  systems  emphasize  com¬ 
prehensive  job  analysis  to  identify 
required  skills,  knowledge  and  abilities 
and  substantial  involvement  of  subject 
matter  experts  in  the  selection  process. 


MSPB  is  also  concerned  that  there  has  been  no 
effective,  systematic  evaluation  of  how  agencies  arc 
administering  the  one  aspect  of  first-line  supervisory 
selection  that  is  mandated  by  law:  supervisory 
probationary  periods.  Obviously,  the  framers  of  the 
Civil  Service  Reform  Act  recognized  the  critical 
importance  of  the  first-line  supervisor’s  job  when 
they  provided  for  this  probationary  period.  The 
period  was  intended  to  serve  as  an  important  final 
test  by  which  agencies  could  evaluate  the  effective¬ 
ness  of  their  selection  actions  (and  save  themselves 
future  problems  caused  by  inappropriate  selections). 
It  can  also  enable  agencies  to  evaluate  their  entire 
supervisory  selection  process.  However,  for  these 
benefits  to  be  fully  realized,  OPM  must  undertake  a 
thorough  evaluation  of  the  use  of  supervisory 
probationary  programs  by  Federal  agencies.  Such 
an  evaluation  would  also  help  answer  important 
questions  raised  by  existing  data  on  probationary 
failure  rates.  For  example,  is  performance  appraisal 
less  rigorous  during  the  supervisory  probationary 
period  and  is  that  period  really  being  used,  as 
intended,  as  the  final  hurdle  in  the  selection  process? 

On  the  research  front,  one  aspect  of  supervisory 
selection  systems  which  calls  for  further  study  is  that 
of  the  quality  of  current  first-line  supervisors  in  the 
Federal  work  force.  A  Federalwide  study  could 
answer  questions  such  as  the  following:  Are  first- 
line  supervisors  doing  a  good  job?  Are  we  getting 
the  highest  quality  supervisors  possible?  Fundamen¬ 
tal  to  these  questions  is  the  issue  of  whether  individ¬ 
ual  agencies  can  really  define  what  “good  supervi¬ 
sory  performance”  means  in  their  organizations. 
And,  once  defined,  do  they  have  any  mechanism  for 
accurately  assessing  good  performance?  As  previ¬ 
ously  indicated,  recent  MSPB  research  on  perform¬ 
ance  management  has  delineated  problems  with  use 
of  formal  performance  appraisals  as  accurate 
indicators  of  performance.  Are  other  measures 
available  to  agencies  for  evaluating  the  quality  of 
supervisory  performance?  If  not,  should  OPM  be 
providing  agencies  with  tools  to  assess  supervisory 
performance,  or  is  this  something  individual 
agencies  must  develop  for  themselves? 
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Finally,  during  the  course  of  this  study,  many 
agencies  expressed  the  desire  to  learn  more  about 
innovative  strategies  available  for  selecting  first-line 
supervisors.  Agencies  have  a  pressing  need  for  a 
vehicle  to  exchange  ideas  and  experiences  in  this 
area.  We  recommend  that  OPM  exercise  leadership 
in  making  better  use  of  the  existing  Interagency 
Advisory  Group  (IAG)  system  as  the  means  for 
identifying  and  disseminating  information  about 
effective  alternatives  in  the  critical  area  of  supervi¬ 
sory  selection. 
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Appendix  1 

U.S.  Marshals  Service’s  Supervisory  Selection  System 


The  U.S.  Marshals  Service  (USMS)  employs  a  multi-phase  assessment  procedure  for  selecting  first-line  Supervi¬ 
sory  Deputy  U.S.  Marshals.  The  first  phase  of  its  system  consists  of: 

■  A  written  application  package  (focusing  on  education,  training,  awards,  and  work  experience) 
which  is  rated  according  to  a  previously  established  crediting  plan; 

■  Annual  performance  appraisal  ratings; 

■  A  physical  fitness  assessment; 

■  A  supervisory  promotion  appraisal  rating  (covering  both  technical  and  supervisory  skills, 
abilities,  and  experiences);  and 

■  A  written  job  knowledge  test. 


In  the  second  phase,  candidates  who  have  been  determined  to  be  highly  qualified  for  a  particular  supervisory 
position  participate  in  an  assessment  center. 

Applicants  interested  in  a  particular  supervisory  position  must  apply  to  the  system  in  order  to  be  considered  for  the 
position.  Scores  from  the  five  components  in  the  initial  phase  are  used  to  determine  which  candidates  wil. 
complete  the  assessment  center  in  order  to  be  further  considered  for  the  position.  As  explained  in  the  Jy  of  this 
report,  the  assessment  center  is  designed  to  provide  additional  information  to  help  selecting  officials  distinguish 
among  highly  qualified  candidates  for  a  position.  Information  concerning  the  candidates’  performance  in  the 
assessment  center  is  sent,  along  with  the  materials  from  the  other  components  of  the  selection  process,  to  a  central 
selection  committee,  the  Career  Development  Board.  The  Board  reviews  materials  submitted,  as  well  as  any 
additional  input  concerning  requirements  for  the  position  provided  by  the  officials  who  have  the  vacancy.  The 
Board  then  makes  a  final  selection  from  among  the  candidates  referred. 
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Appendix  2 

Mine  Safety  and  Health  Administration’s 
Supervisory  Pool  Program 


This  program,  which  was  developed  for  the  Supervisory  Coal  Mine  Safety  and  Health  Inspectors  of  the  Mine 
Safety  and  Health  Administration  (MSHA),  is  predicated  on  the  philosophy  that  candidates  should  be  fully  trained 
prior  to  being  placed  in  supervisory  positions.  Candidates  are  selected  for  a  supervisory  development  pool,  and 
engage  in  a  1-year  course  of  instruction  which  combines  formal  classroom  training  and  on-the-job  developmental 
assignments.  Supervisory  jobs  in  MSHA’s  10  coal  districts  across  the  United  States  arc  subsequently  filled  by 
successful  pool  graduates. 

I'he  vast  majority  of  applicants  to  the  program  meet  basic  qualification  requirements,  and  panels  of  subject  matter 
experts  arc  used  to  further  evaluate  applicants  against  job-related  factors.  The  lop  candidates  from  this  process, 
the  highly  qualified  group,  go  through  a  final  evaluation  phase  which  includes  a  panel  interview  conducted  by 
three  managers.  Recently,  MSHA  added  an  assessment  center  to  the  final  evaluation  phase.  (S:ncc  the  current 
supervisory  p<x)l  is  the  first  to  have  experienced  the  assessment  center,  and  they  will  not  be  candidates  for 
,  .rmanent  supervisory  jobs  until  at  least  the  end  of  1989,  MSPB  did  not  examine  MSHA’s  perceptions  regarding 
the  success  of  this  new  component.)  The  results  of  this  final  phase  arc  used  to  rank  the  candidates  for  certification 
to  the  MSHA  Coal  Administrator,  who  makes  all  selections  for  the  training  program. 

Candidates  selected  for  the  pool  attend  26  weeks  of  technical  and  management-related  classes  at  MSHA’s 
National  Mine  Health  and  Safely  Academy  at  Bccklcy,  West  Virginia.  Classes  at  the  Academy  arc  generally 
divided  into  4-wcck  sessions,  each  followed  by  a  4-wcck  field  assignment.  Evaluation  tools  such  as  examinations, 
exercises,  and  reports  arc  used  to  monitor  candidates’  successful  completion  of  each  phase,  with  individualized 
course  work  assigned  to  overcome  identified  weaknesses.  Upon  completing  program  requirements,  successful 
candidates  arc  placed  on  an  unranked  register  and  have  the  opportunity  to  be  considered  for  vacancies  as  they 
occur. 

A  memorandum  to  pool  members  alerts  them  when  a  specific  job  is  being  filled.  Interested  members  can  request 
that  their  names  be  considered  for  the  vacancy.  If  no  members  indicate  interest  when  the  vacancy  is  first  identi¬ 
fied,  members  of  the  pool  arc  surveyed  a  second  lime  to  determine  interest.  If  no  candidates  for  the  position  arc 
found  in  this  second  survey,  the  names  of  all  members  of  the  pool  arc  submitted  to  the  selecting  official  for 
consideration.  Depending  upon  the  location  of  the  supervisory  job,  a  selected  candidate  may  have  to  relocate 
when  accepting  the  job  offer.  Program  officials  have  planned  for  this  possibility  by  requiring  signed  mobility 
agreements  from  supervisory  pool  candidates  before  they  can  enter  the  1-ycar  training  and  development  program. 

Since  inception  of  the  program  in  1984,  MSHA  has  announced  and  filled  three  supervisory  pool  classes.  (Class 
size  and  timing  have  varied  based  upon  projected  supervisory  turnover,  but  generally  the  goal  has  been  to  gradu¬ 
ate  sufficient  candidates  to  comprise  a  register  which  can  be  used  for  about  2  years.)  Competition  for  this  training 
and  development  opportunity  has  been  keen;  a  total  of  429  employees  have  applied,  from  which  72  have  been 
selected  for  participation. 
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Appendix  3 

Federal  Aviation  Administration’s 
Supervisory  Identification  and  Development  Program 


In  the  early  1980’s,  the  Federal  Aviation  Administration  (FAA)  designed  a  new  system  to  identify,  develop,  and 
select  employees  for  first-line  supervisory  positions  in  Air  Traffic  Control.  A  task  force  consisting  of  employees 
in  the  Air  Traffic  work  force,  as  well  as  field  and  headquarters  human  resource  management  personnel,  was 
established  to  develop  the  system.  Initially,  the  system  was  tested  in  two  regions,  Northwest  Mountain  and 
Southwest,  and  covered  only  Air  Traffic  Control  supervisory  positions.  The  program  has  now  been  expanded  to 
the  national  level  and  covers  all  of  FAA’s  regions  and  includes  other  functional  areas  in  addition  to  Air  Traffic 
Control. 

Between  1985-1987,  in  the  two  test  regions,  over  2,100  employees  applied  to  the  Supervisory  Identification  and 
Development  Program  (SIDP).  During  the  initial  administration  of  SIDP  at  the  national  level  in  1988,  over  5.900 
employees  applied.  In  all  nine  regions  in  which  SIDP  currently  operates,  all  applicants  for  any  first-line  air  traffic 
supervisor  position  must  apply  to  the  program  (including  current  supervisors  applying  for  different  first-line 
supervisor  positions).  Applications  for  the  SIDP  are  typically  accepted  once  per  year. 

The  identification  and  development  process  consists  of  three  steps,  as  follows: 

Step  1:  Peer  and  Supervisory  Assessment  -  The  applicant  submits  the  names  of  three  to  seven  (depending  on  the 
size  of  the  facility)  co-workers  (peers)  at  his  or  her  facility  who  have  agreed  to  rate  the  applicant  on  four  perform¬ 
ance  dimensions:  Leadership,  Interpersonal,  Communications,  and  Professional  Competence.  The  applicant’s  im¬ 
mediate  supervisor  also  rates  the  applicant  on  the  four  dimensions.  (The  applicant  may  also  include  one  other 
facility  supervisor  as  a  potential  rater  in  the  place  of  one  of  the  peers.) 

Ratings  made  by  the  peers  and  supcrvisor(s)  arc  combined  on  each  dimension  to  produce  one  rating  on  each 
dimension  and  an  overall  rating.  For  each  applicant  pool  (i.e. ,  job  series),  applicants  within  the  region  are  then 
ranked  according  to  their  overall  rating.  The  top  applicants  within  each  pool  arc  referred  for  the  next  step  in  the 
SIDP  process.  (The  actual  number  referred  for  the  next  step  depends  on  projected  vacancies  for  the  region  and 
percentage  of  candidates  expected  to  make  it  through  the  remaining  steps  of  the  SIDP  process.) 

Step  2:  Skill-Based  Interview  -  The  Skill-Based  Interview  is  actually  a  combination  of  face-to-face  interview 
and  simulation  exercises  in  which  candidates  participate.  The  candidates’  performance  in  the  interview  and 
exercises  is  rated  by  a  trained  panel  of  observers.  The  exercises  are  intended  to  simulate  situations  that  candidates 
might  actually  encounter  in  the  supervisory  job.  Candidates  arc  given  both  verbal  and  written  feedback  concern¬ 
ing  their  performance  in  the  Skill-Based  Interview,  including  both  strengths  and  weaknesses  identified. 
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Step  3:  Placement  on  a  List  -  Candidates’  performance  on  the  Skill-Based  Interview  determines  which  one 
of  three  avenues  they  will  follow  in  the  SIPP  process: 

1)  Candidates  who  perform  very  effectively  are  placed  on  an  Eligible  For  Consideration  (EFC)  list.  Lists  are 
maintained  in  each  region  by  an  SIDP  manager  for  the  region.  Each  time  a  position  becomes  vacant,  candidates 
on  the  EFC  list  are  notified  about  the  vacancy.  All  of  those  expressing  an  interest  are  referred  in  unranked, 
alphabetical  order,  to  the  selecting  official  for  consideration.  The  selecting  official  receives  a  package  of  informa¬ 
tion  on  all  referred  candidates.  The  package  includes  the  profile  of  peer  and  supervisory  ratings,  a  report  on  the 
candidate’s  performance  in  the  Skill-Based  Interview,  and  developmental  and  work  history  information. 

2)  Candidates  who  perform  at  an  ineffective  level  in  the  Skill-Based  Interview  are  counseled  on  strengths  and 
weaknesses,  are  encouraged  to  continue  self-development,  and  may  reapply  to  SIDP  later.  If  interested,  candi¬ 
dates  are  given  assistance  in  determining  ways  to  improve  identified  weaknesses. 

3)  Candidates  who  perform  at  a  moderate  level  of  effectiveness  are  referred  to  a  candidate  review  board.  This 
board  then  decides  whether  they  should  receive  formal  development,  and  using  an  Individual  Development  Plan, 
have  priority  in  using  training  resources,  or  whether  they  should  be  encouraged  to  pursue  self-development  and 
reapply  to  SIDP  later.  The  progress  of  candidates  given  formal  development  is  followed  by  local  personnel 
officials  as  well  as  the  board.  After  these  candidates  have  completed  their  Individual  Development  Plans,  the 
board  decides  whether  to  place  them  on  an  EFC  list,  or  to  have  the  Skill-Based  Interview  panel  reassess  perform¬ 
ance  in  areas  previously  identified  as  weaknesses. 


A  REPORT  BY  THE  U.S.  MERIT  SYSTEMS  PROTECTION  BOARD 


FIRST-LINE  SUPERVISORY  SELECTION  IN  THE  FEDERAL  GOVERNMENT 


23 


Appendix  4 

Department  of  the  Army’s 
Civilian  Career  Evaluation  System 


In  the  early  1980’s,  the  Department  of  the  Army,  in  a  joint  effort  with  the  Office  of  Personnel  Management,  began 
designing  the  Army  Civilian  Career  Evaluation  System  (ACCES),  an  automated  centralized  candidate  evaluation 
and  referral  system  to  be  used  in  filling  positions  in  its  civilian  career  programs.  While  some  other  agencies  have 
referral  and  selection  systems  which  are  centralized  and  have  automated  features.  Army  wanted  to  develop  a 
computer-assisted  evaluation  system  which  could  enable  both  sound  measurement  of  applicants  as  well  as  ease 
and  efficiency  of  operation.  To  meet  this  objective.  Army  conducted  extensive  job  analyses  for  the  job  series  they 
intended  to  cover  under  ACCES.  The  job  analyses  identified  rating  elements  (knowledge,  skills,  or  abilities — 
KSA’s)  to  be  used  in  the  evaluation  and  referral  process  for  these  job  series.  The  job  analyses  also  identified 
rating  elements  to  be  used  specifically  when  filling  supervisory  positions. 

Jobs  filled  under  ACCES  are  grouped  according  to  career  programs.  The  various  job  series  included  within  a 
particular  career  program  represent  the  functional  areas  covered  by  that  field  of  work.  For  example,  the  Civilian 
Personnel  Administration  Career  Program  consists  of  job  series  such  as  GS-221  (Position  Classification),  GS-212 
(Personnel  Staffing),  GS-230  (Employee  Relations),  and  GS-233  (Labor  Relations),  each  of  which  represents  a 
function  found  in  the  personnel  field.  A  separate  inventory  of  rating  elements  exists  for  each  of  the  career 
programs  covered  by  ACCES,  and  each  inventory  includes  KSA’s  identified  for  supervisory  positions.  Seventeen 
career  programs  will  eventually  be  covered  by  ACCES;  there  are  currently  6  career  programs  which  are  fully 
operational  and  comprehensive  job  analyses  have  been  completed  for  1 1  other  programs  which  are  scheduled  for 
implementation  shortly. 

Each  functional  area  within  a  career  program  has  a  set  of  “core”  rating  elements  or  criteria  that  must  be  used  in 
evaluating  applicants.  Additionally,  since  duties  for  individual  jobs  within  a  career  program  can  differ,  the 
selecting  official  can  identify  other  relevant  rating  elements  to  be  used  in  filling  a  particular  job.  For  example, 
when  filling  a  Personnel  Staffing  Specialist  job,  the  selecting  official  must  use  the  elements  established  for  the 
staffing  function.  However,  the  official  may  also  include  some  elements  established  for  the  employee  relations 
function,  if  they  happen  to  be  applicable  to  the  job  he  or  she  is  filling.  Additionally,  the  selecting  official  has  an 
opportunity  to  weight  the  importance  of  the  rating  elements  which  will  be  used  in  determining  the  referral  scores 
for  applicants  who  express  an  interest  in  the  job. 

An  employee  wishing  to  be  considered  for  job  referrals  under  ACCES  must  submit  an  application  package 
containing  ratings/documentation  on  the  rating  elements  established  for  the  specific  career  program.  The  follow¬ 
ing  materials  comprise  the  major  components  of  the  application  package: 

1.  Self  ratines.  The  applicant  rates  himself  or  herself  on  all  of  the  knowledges  identified  as  relevant  to  the  career 
program. 

2,  Supervisor’s  ratines.  The  applicant’s  immediate  supervisor  rates  the  applicant  on  all  of  the  knowledges 
established  for  the  career  program.  The  supervisor  also  rates  the  employee  on  all  of  the  abilities  identified  as 
important  for  the  career  program. 
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3.  Accomplishments.  The  applicant  writes  brief  summaries  of  personal  accomplishments  which  exemplify  the 
abilities  identified  as  important  for  the  career  program.  This  documentation  is  evaluated  (scored)  by  a  panel  of 
trained  raters,  who  are  unaware  of  the  applicant’s  identity. 

Therefore,  each  applicant  has  two  sets  of  knowledge  ratings  (those  provided  by  the  applicant  and  those  provided 
by  the  supervisor),  and  two  sets  of  ratings  on  abilities  (those  provided  by  the  supervisor  and  those  based  on  the 
panel  evaluation). 

4.  Work/education/training  history.  The  applicant  is  given  an  opportunity  to  briefly  describe  past  work, 
education  and  training  experiences.  This  information  is  made  available  to  the  selecting  official  after  the  applicant 
has  been  referred  for  a  particular  position,  in  order  to  provide  additional  information  on  which  to  base  the  selec¬ 
tion. 

5.  Referral  desires  and  geographic  availability.  The  applicant  provides  information  concerning  the  types  of 
jobs  (i.e.,  job  series  and  functional  areas)  and  grades  for  which  he  or  she  wants  to  be  considered.  The  applicant 
also  indicates  whether  he  or  she  is  interested  in  supervisory  positions.  Additionally,  the  applicant  is  asked  to  note 
geographical  areas  in  which  he  or  she  would  accept  a  position. 

AH  of  the  information  provided  by  the  applicant  (except  for  the  work/education/training  history)  is  entered  into  an 
automated  applicant  data  file.  When  there  is  a  job  vacancy  to  be  filled,  information  concerning  the  job  (e.g.,  job 
scries,  geographic  location,  rating  elements  and  their  respective  weights)  is  entered  into  the  ACCES  computer. 

The  computer  compares  this  job  information  with  applicant  information  which  is  already  on  file,  in  order  to  match 
interested  applicants  with  the  vacancy.  This  initial  matching  is  done  primarily  on  the  basis  of  applicants’  ex¬ 
pressed  interest  in  the  type  of  position  (i.e.,  job  scries,  supervisory  versus  nonsupervisory),  grade,  and  geographi¬ 
cal  location  of  the  job  being  filled.  Once  the  initial  match  is  made,  composite  referral  scores  arc  computed  on 
each  of  these  matched  applicants.  An  applicant’s  composite  score  is  based  on  the  two  sets  of  knowledge  ratings 
and  the  two  sets  of  ability  ratings  which  were  previously  mentioned.  Since  selecting  officials  tailor  the  rating 
criteria  and  their  weights  to  each  vacant  position  they  fill,  there  is  a  high  probability  that  an  applicant’s  referral 
score  will  change  from  one  vacancy  referral  to  the  next. 

Selecting  officials  may  request  a  list  of  candidates  eligible  for  promotion,  and  one  of  two  types  of  lists  of  candi¬ 
dates  eligible  for  lateral  (same  grade)  assignments.  If  a  list  of  promotion  cligibles  is  requested,  the  lop  candidates 
(based  on  their  composite  referral  scores)  are  alphabetically  placed  on  a  list  that  is  sent  to  the  selecting  official. 
(The  number  of  top  candidates  to  be  referred  varies  among  career  programs,  and  ranges  between  10  and  50 
candidates.)  Depending  upon  the  type  of  lateral  list  the  selecting  official  has  requested,  he  or  she  receives  cither  a 
list  of  all  applicants  eligible  for  lateral  assignments,  or  a  list  of  only  the  top  applicants  (based  on  their  composite 
scores).  Regardless  of  the  type(s)  of  list(s)  requested,  a  printout  of  the  ratings  on  the  relevant  elements  and  the 
work/education/training  history  form  for  each  referred  candidate  accompanies  the  referral  list. 

Candidates  arc  notified  by  mail  to  their  home  or  office  addresses  that  they  have  been  referred  for  a  particular  job. 
They  are  asked  to  indicate  to  the  personnel  office  servicing  the  organization  with  the  vacancy  whether  they  arc 
interested  or  not  interested  in  the  specific  vacancy.  After  the  selecting  official  learns  of  who  on  the  list  is  inter¬ 
ested  in  the  vacancy,  he  or  she  reviews  information  received  on  each  candidate,  and  may  choose  to  conduct 
follow-up  interviews  (in  person  or  by  telephone)  or  ask  for  other  information  (e.g.,  copies  of  previous  performance 
appraisals)  before  making  the  final  selection. 
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